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Over the last two decades, state and federal 
laws and grant programs, such as state 
accountability polices, the No Child Left 
Behind Act (NCLB),Race to the Top, Tide I 
School Improvement Grants, and State 
Longitudinal Data System Grants, have given 
state education agencies (SEAs) considerably 
more responsibilities for directing and 
guiding the improvement of low-performing 
schools. At the same time, they have pressed 
SEAs and school districts to incorporate 
research-based school improvement policies 
and practices in their statewide systems of 
support for low-performing schools, 
technical assistance for districts, professional 
development for teachers, and school 
improvement programs. Policymakers have 
urged SEAs to engage with organizations 
external to their own agencies to extend 
their strained capacity, and to help them 
collect and use research or other evidence 
(see, for example, Rennie Center, 2004) . A 


variety of organizations involved in this 
enterprise have emerged over the last two 
decades (Rowan, 2002). For example, the 
2002 authorization of the Elementary and 
Secondary Education Act’s (ESEA) 
comprehensive assistance centers was 
specifically designed to provide and 
encourage SEAs use of research. 

Although studies of districts’ and schools’ use 
of research exist (see for example, Coburn, 
Honig, & Stein, 2009; Daly & Finnigan, 
2011; Farley-Ripple, 2012), we know little 
about how SEAs search for, select, and use 
research and other kinds of evidence in their 
school improvement strategies. While one 
might assume similarities in research use 
behaviors, both the organizational structures 
of SEAs and the population of external 
organizations with which they interact are 
quite different than schools and districts, and 
the most recent in-depth study of SEAs was 
conducted nearly 20 years ago (Lusi, 1997). 
The exploratory study on which this brief is 
based was designed to fill that gap by 
examining: 1) where SEA staff search for 
research, evidence-based, and practitioner 
knowledge related to school improvement; 

2) whether and how SEA staff use research 
and these other types of knowledge to 
design, implement, and refine state school 
improvement policies, programs and 
practices; and, 3) how SEAs are organized to 
manage and use such knowledge (Goertz, 
Barnes, Massell, Fink, & Francis, 2013). 

To clarify the nature of evidence that SEAs 
sought and used, we distinguished among 
three types of knowledge in our data 
collection and analysis. We define research- 
based knowledge as research findings that have 
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been to varying degrees “collated, summarized, and 
synthesized,” and then are presented in ways that 
provide empirical or theoretical insights or make 
them otherwise informative (Davies & Nudey, 2008). 
We include in this category published original 
research, research syntheses, summaries or meta- 
analyses, and evaluation reports. We also consider 
forms of research-based knowledge that are designed 
for use in practice; that is, models, programs, protocols or 
other tools that embed research or research-based 
practices in guides to action. Older knowledge 
utilization models assumed that simply transmitting 
such knowledge to policymakers or practitioners 
would be sufficient to create change. But new 
models show that research-based knowledge is not 
sufficient to meet the needs of professionals using it. 
Integrating contextual, local, and practitioner 
knowledge with research knowledge is critical to 
developing “useable” knowledge to guide action 
(Honig & Coburn, 2008; Hood, 2002; Huberman, 
1990; Lindblom & Cohen, 1979; Weiss, Murphy- 
Graham, Petrosino, & Gandhi, 2008). Therefore, we 
considered how the SEAs incorporated other evidence- 
based knowledge, which we define as data, facts, and 
other information relevant to the problem of school 
improvement, such as formative feedback loops on 
implementation, and practitioner knowledge, which is 
the information, beliefs, and understanding of context 
that practitioners acquire through experience, along 
with research in their decision-making processes. 

The formal organizational structure of most SEAs has 
long been criticized for its hierarchical and segmented 
nature, and for its focus on compliance instead of on 
guidance and support for meaningful improvements 
in schools or districts. In her research study of two 
SEAs, Lusi (1997) argued that flatter, less segmented 
management structures could help build internal and 
external connections and produce the kind of 
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adaptive organization that would be more conducive 
to coherent improvement policies and the flow of 
knowledge. More recent studies in other settings 
confirm that flexible professional connections across 
traditional organizational boundaries improve 
problem-solving using varied but relevant expertise 
(Dutton & Heaphy, 2003;Weick & Sutcliffe, 2001; 
Wenger, McDermott, & Snyder, 2002). Sociologists 
have long studied these kinds of connections, known 
as “social networks,” to understand the diffusion of 
knowledge and innovation within and across 
organizations, including more recent studies of 
schools and districts (Daly, 2010). A few researchers 
have used social network theory and methods to 
study state education policy networks (Miskel & 

Song, 2004; Song & Miskel, 2005). We applied social 
network perspectives and methods here to examine 
SEA’s communication structures, search and 
incorporation networks, and network properties, 
such as social capital, and to identify the most 
central “knowledge brokers” or influential 
knowledge sources in the research and other 
knowledge networks. 

Our study included three SEAs that are located in 
different regions of the country and vary in size (from 
250 to 500 staff), organizational structure, and school 
improvement strategies. Data for the study were 
collected between 2010 and 2012. We conducted in- 
depth interviews with high-level SEA staff involved 
directly in school improvement and in related 
programs (e.g., curriculum and instruction, 
accountability, special programs, teacher policy) and 
with a small number of leaders of external 
organizations that were central to research use in the 
SEAs. We also sent a web-based survey to all 
professional staff in the two smaller SEAs, and to all 
staff working in school improvement and related 
departments and a representative sample of other 
professional staff in the third SEA. Our analyses are 
based on a total of 62 interviews and 300 surveys in 
the three SEAs 1 , as well as documents describing SEA 
school improvement policies and tools designed for 
district and school use. 

Respondents identifed the offices, organizations and 
individuals they turned to both within and outside 
their SEA when communicating about work, and 


1 All survey respondents were asked whether their work related “in any way to improving low-performing schools and school districts” in their state. We used 
results from all respondents who answered “yes” to this question; that is, staff who self-identified as being involved in school improvement work regardless of the 
SEA office in which they worked. 
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more specifically, when seeking research, data, and 
practitioner advice on programs and practices targeted 
at improving low-performing schools and school 
districts. 2 We used the survey data to identify internal 
and external sources of research and other types of 
knowledge, to analyze the size, strength, and 
configuration of the four networks, including patterns 
of cross-office or within-office communication, and 
to identify the most highly connected knowledge 
brokers and influential individuals, offices, or 
organizations in the networks. We drew on the 
direct, interpersonal networks of these influential 
individuals along with interview data to corroborate 
and interpret the broader network analyses, and to 
provide more specific information on the internal and 
external sources of research or other knowledge. 
Interview data and document reviews also provided 
more detail on the types and qualities of research or 
other forms of knowledge that SEA staff sought and 
used in school improvement decision-making, the 
research incorporation process, and institutional, 
political, or other factors that influenced research 
search, incorporation, and use. 

The findings presented in this brief focus on how 
SEAs acquired and used research knowledge in their 
school improvement policies and programs. We did 
not examine the effects of their use of research or 
other evidence on improving school or student 
outcomes. Because we focused on research use for 
school improvement in a small number of states, our 
findings may not generalize to other SEAs or other 
education policy areas. Our study, however, is the 
first to systematically map information networks 
within SEAs and between SEA staff and external 
sources of support, and provides important insights 
into how SEA staff search for and incorporate 
research in their work. 

The Structure and 
Strength of SEA 
Research Networks 


Multiple SEA staff in the three study states actively 
searched for and were receptive to research ideas and 


related information from both within and outside 
their agencies. About 75% of the staff in each 
agency asked their SEA colleagues for research 
advice, while a little less than one-third turned 
to external organizations or individuals for 
similar information. In each SEA, some, but 
not all, of these staff named multiple colleagues, 
offices and external organizations as sources of 
research information. 

Contrary to the usual image of SEAs as “siloed” 
organizations, our network analyses showed 
considerably more cross-office connections than we 
anticipated given the literature on SEA structures. 
Respondents attributed cross-department 
communications to multiple factors, including state 
and federal accountability demands, competition for 
federal grants that required integrated proposals, 
reduced SEA staffing, and SEA leaders committed to 
more collaborative organizational cultures. For 
example, special education staff in State C perceived 
that NCLB accountability requirements had brought 
them into school improvement meetings in an 
unprecedented way. Other federal initiatives, such as 
the Race to the Top grants, stimulated cross-office 
search and exchange of research-based ideas in State 
B. Leadership in all three SEAs also facilitated cross- 
agency collaboration by creating cross-office teams 
to share information or work on common tasks or 
problems. States B and C were active participants in 
an Academy of Pacesetting States hosted by the 
Center on Innovation and Improvement (CII), a 
national comprehensive assistance center that 
convened cross-office state teams on a regular basis 
to share and discuss research or other topics related 
to statewide systems of school improvement. State A 
established cross-departmental task forces to design 
its system of tiered-intervention and to manage and 
monitor performance on the SEA’s major goals. 

These broad, more informal cross-department and 
external connections facilitated the flow of research 
information and new ideas, but internal cross- 
department connections were weaker than those 
within departments. This suggests that colleagues 
within the same department or office engaged in 
more frequent interactions, and these had a greater 
impact on individuals’ work than did the cross- 


2 We were unable to collect the names of individuals from whom the respondents sought research or other types of advice and information in 
State A and use office or department names instead. 
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department connections. Thus, formal 
organizational structures still delineate many 
functional responsibilities and lines of communication. 

While the school improvement research networks in 
each state included an array of staff from across the 
SEAs and numerous external organizations, only a 
small number of individuals, offices and external 
partners were central actors in these networks. We 
identified the key actors or offices by mapping each 
participant’s location in the networks (e.g., central or 
more peripheral), as well as the strength 3 and 
direction 4 of their connections to each other. 
Influential and well-connected individuals and 
offices tended to be clustered more centrally in the 
core of the network maps. 5 “Influential” 
participants are those who were highly sought after 
for research information; that is, they were 
mentioned as a source of advice by many SEA staff. 
“Well-connected” participants are those who both 
sought research ideas and information from a range 
of sources, and, at the same time, provided 
information to a range of SEA colleagues, thus 
serving as knowledge brokers in the research 
networks. These two sets of individuals or offices 
tended to overlap considerably. Figures 1-3 below 
show the structure of the three SEA research 
networks, and the staff, offices and external 
organizations involved in, and their location within, 
each network. 

The directors of school improvement were the most 
influential and well-connected SEA staff in the 
research networks in States B and C (Figures 1 and 
2). These directors and some of their staff (green 
nodes) were connected with several other salient 
offices related to school improvement. In State B, 
these key participants included leadership from the 
Assessment and Accountability Department (grey 
nodes), including staff in the Research and 
Evaluation Unit (red nodes), the Commissioner’s 
and Deputy Commissioner’s Offices related to 
academic matters (yellow nodes), ESEA program 


monitoring (aqua nodes), and a consultant from 
special education (pink node) . We also found strong 
connections between the Curriculum and 
Instruction Office (dark green nodes) and the 
School Improvement Office, both of which were 
within the School Improvement Department. Two 
external organizations — a statewide professional 
membership association and the state’s regional 
comprehensive assistance center (black shapes) — 
were centrally located and had stronger connections 
to the School Improvement Department than most 
other organizations in State B’s research network, 
although the CII and a state university were also 
quite influential. 

School improvement and curriculum and 
instruction are housed in different departments in 
State C, and primarily leadership from both 
departments, along with very few staff, were at the 
center of its research network. While the Assessment 
Offices were not as prominent as they were in State 
B, directors of a very small Research Office housed 
in the Commissioner’s Office and ESEA program 
monitoring were highly influential in the network. 
Thus, like State B, most of the influential internal 
brokers were in formally designated leadership roles. 
Also like State B, a few external organizations were 
prominent in the research network — the CII, State 
C’s regional comprehensive assistance center, and a 
statewide professional membership association. 

In contrast to the other states, the Research 
Department was the most influential in State A 
(Figure 3) . It resided at the center of State As 
research network, with multiple connections to 
most other departments in its SEA. This relatively 
large office plays multiple roles in its SEA: 
conducting research reviews for offices throughout 
the agency; helping program offices design, procure, 
and manage program evaluations; preparing data 
reports and briefings for accountability review 
teams; and, developing analytical tools. Some of this 
work is done in-house, and some, particularly 


3 Strength of network connections was measured though a combination of the reported frequency of communication about research or 
other kinds of information, and the influence respondents perceived the resulting information to have on their work. Degree of strength 
was considered using a matrix ranging from a cell defined by highly influential/ daily contact (200) to a cell defined by not influ ential/a few 
times per year contact (0.5). 

4 An individual or office can be the seeker of research information (considered an “out-tie”) or the named source of research information 
(considered an “in-tie”). To examine ties between SEA members and external members of networks, we used “out-ties” from our SEA 
respondents forming in-ties to external organizations as we interviewed a subsample, but did not survey, external organizations. 

5 We also identified the most central staff or offices using rank ordered standardized centrality measures showing the percentage of all possible 
ties directed into sources of research (in-ties) as well as out from those seeking information (out ties) . 
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program evaluations and literature reviews, is 
contracted out. But the most well-connected 
research knowledge broker was the Accountability 
Department that conducts formal reviews and 
monitors the performance of low-performing school 
districts. The School Improvement Department and 
its offices, along with the Curriculum and 
Instruction Department, were also highly influential 
hubs of activity. Similar to State C, curriculum and 
instruction and school improvement are housed in 
separate departments, but though influential, the 
Curriculum and Instruction Department did not 
have many direct research connections with school 


improvement in State A. Finally, perhaps due to the 
role of the Research Office, external organizations 
remained at the periphery of State As research 
network. 


Figure 1 

State B, Research Network, Strength of Ties 



CONSORTIUM FOR POLICY RESEARCH IN EDUCATION 


5 





Figure 2 

State C, Research Network, Strength of Ties 
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Figure 3 

State A, Research Network, Strength of Ties 
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External Sources for 
Research Knowledge 


Although fewer SEA actors turned to external 
sources of research on school improvement, these 
organizations played a key role in the research that 
policymakers accessed and used. The SEA staff 
named a large and diverse array of external 
organizations — between 37 in State A and 42 in 
State B. These organizations were invited to present 
new ideas about school improvement, provided or 
collaboratively developed research-based tools and 
strategies, conducted evaluations, synthesized 
research on relevant topics, and more. As we 
discussed above, a subset of these external 
organizations played central roles in SEA research 
networks in States B and C. 

The most commonly mentioned source of external 
research information was the federal government. 
(See Figure 4.) SEA staff turned to federal agencies, 
including several offices within the U.S. Department 
of Education, and federally funded centers, such as 
the ESEA comprehensive assistance centers, regional 
education labs, and other technical assistance centers 
that support the implementation of more targeted 
educational programs (e.g., special education or 
vocational education). The second largest external 
source of research advice was professional 
membership associations. These included: 1) 
associations focused on specific subject matter, or 
teaching and learning more generally, such as state 
and national reading, mathematics and technology 
organizations, and theASCD;2) occupationally 
focused associations, such as the Council of Chief 
State School Officers or state-level associations 
representing superintendents, principals, and federal 
program administrators; and, 3) regional 
organizations, such as the Southern Regional 
Education Board. Staff in the three SEAs, however, 
reached out primarily to national, not state, 
professional membership associations for research 
advice. Fewer SEA staff sought research from 
institutions of higher education or research 
organizations, although one to two universities did 
play an integral role in school improvement designs 
and delivery in States B and C, and SEA staff in 
State A often turned to university faculty for help on 
specific projects. SEA staff also turned to a different 


mix of federal organizations across the three states. 
For example, State A sought research advice more 
ffequendy from various regional education labs than 
staff in States B and C who relied more heavily on 
the regional and content centers within the 
comprehensive assistance center network. 

Figure 4 

External Sources for Research Knowledge 


Govt: Federal 
Govt: State/regional/local 
Professional Membership 
Higher Education 
Research/Provider (NGO) 
Education Journals 
Other/Unknown 



■State A 
■ State B 
State C 


0% 10% 20% 30% 40% 50% 


This variation in the states’ external research 
networks reflected differences in stages of policy 
development, internal capacity, the structure of 
external environments, and prior partnership 
histories. For example, the basic components of the 
State As school improvement system had been in 
place for many years; the overarching design was not 
a target for major review. As a result, State A was 
more likely to pull in a range of external partners, 
on an as-needed basis, to co-develop very discrete 
and specific tools and resources. This process was 
facilitated by the SEA’s robust research office. State 
C, in contrast, was bracing for more and more 
schools to come within its purview for not meeting 
state or federal accountability standards in the midst 
of a very spare and declining SEA workforce. With 
limited research capacity and expertise in school 
improvement, SEA staff turned to the CII, as well as 
their own regional comprehensive assistance center 
and a state professional membership association, to 
help them redesign their supports and create a 
research-based infrastructure of tools to monitor and 
assist schools. Their engagement with these 
organizations was frequent and extended over many 
years. Similarly, having very limited capacity for 
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high school improvement and guidance, States B and 
C sought assistance from the National High School 
Center. State C, for example, used this Centers 
research and the work of a consulting group to 
develop an “early warning system” to identify 
students at risk of dropping out of school. 

Much of the external search was driven by personal 
connections and prior work histories. While a few 
of the SEA staff who we interviewed suggested they 
used a range of Internet and academic resources, 
many turned to their existing network of academics 
and/ or memberships and affiliations with education 
organizations to access research and new ideas. SEA 
staff also turned to organizations with which the 
SEA had a history of doing work. For example, 
regional area education agencies and their respective 
membership association have long partnered with 
the SEA in State B in delivering services to schools. 
These regional government agencies pioneered 
strategies for improvement that State B later adopted 
as part of its statewide systems of support, such as a 
design for working with low-performing schools to 
model an improvement process. State C, which did 
not have an equivalent set of regional government 
partners, turned instead to a state-level professional 
membership association with which they had long 
worked in school improvement design and delivery. 
Organizations that have a previous work history 
with SEAs are attractive sources of research-based 
knowledge because they often know the local 
context in which the SEA is working, are familiar 
with the strengths of SEA staff, and are viewed as 
credible sources of information. 

Some external sources of research were also viewed 
as more neutral purveyors of knowledge than the 
SEA, an asset if targeted schools were wary of the 
state reform agenda. Similarly, the expertise carried 
by external organizations or individuals could 
provide an outsiders’ perspective on whether state 
efforts were within the bounds of best practice — an 
important metric for state agencies and policymakers 
who are engaged in often uncertain work. For 
example, when State A undertook a review of its 
district and school improvement standards, it asked 
its regional education lab to identify the research 
underlying the proposed standards and their impact 
on schools. The SEA then compiled this research 
into a guide and posted it on its website so that, as 


one staff person noted, “people understand they’re 
being held accountable to things that research tells 
us are important. But also so that... we can tie the 
assistance around some of what research is telling 
us.” State C sought out the advice and engagement 
of a university faculty member who had been a 
well-respected former superintendent, recognizing 
that his support would open the doors, and the 
minds, of other superintendents. 

Finally, external organizations frequently played an 
important role in synthesizing and packaging 
research to make it useable, and useful, to SEA work. 
Many of the SEA staff we interviewed expressed a 
desire to know more about research, but it was 
difficult for them to find time to stay current with 
the literature and to incorporate research directly 
into their school improvement work. An 
administrator in State A explained, “Because out- 
work [in my office] is so huge, I’ve relied on 
consultants and organizations that can capture and 
summarize... research so that we can figure out, 
focus on how we’re going to use it to inform out- 
work.” As we describe in the following sections, the 
more influential external organizations in SEAs’ 
networks brokered, jointly developed with states, or 
helped states adapt research-based, but useable tools 
that translated research into more specified guides 
for action. 

Incorporating Research 
Within Core Networks 

While broad, but relatively weaker cross-department 
and external research networks facilitated the search 
process and the flow of diverse ideas in the SEAs, a 
set of well-connected, influential SEA staff brought 
research and other kinds of information from these 
different sources into stronger, smaller working 
groups that collectively addressed problems of school 
improvement. We conceptualized these groups as 
“core networks .” 6 These groups, which generally 
included leaders and other staff within school 
improvement offices, a few key external 
organizations and, in a very few cases, colleagues 
from other departments in the agency, enabled key 
SEA staff, who were typically central in the 
practitioner as well as research networks, to actually 


6 Here we draw on work ofWenger, McDermott, and Snyder (2002) who studied “communities of practice” in other settings. 
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incorporate research into strategies that were 
workable in the context of their respective states. 

As research has shown in other settings (see, for 
example, Barnes, Camburn, Sanders, & Sebastian, 
2010; Honig & Coburn, 2008; Spillane, Reiser, & 
Reimer, 2002), SEA staff incorporated research into 
their school improvement strategies through a 
distinctly social process in which the network 
members interpreted, challenged, and otherwise 
made sense of research over time. During this 
incorporation process, the core network groups used 
local practitioners’ feedback, state professionals’ 
experience, and external partners’ knowledge of 
relevant research to contextualize various research 
findings in light of their states’ school improvement 
needs. In contrast to models of research 
dissemination in which generalized, primarily 
decontextualized findings advanced by researchers 
are transmitted to users, in these core networks, users 
and a few providers worked collectively to adapt 
research to address particular problems and, in some 
cases, to co-construct new useable knowledge for 
guiding action. 

One characteristic of the core networks was that a 
group of people came together to address common 
problems and goals. Key school improvement staff 
in States B and C who had developed strong ties to 
key liaisons in the federal comprehensive assistance 
center system were able to learn about relevant 
research and how to apply it from developers of 
school improvement models or promising practices 
rooted in that research, and with states who were 
puzzling over similar problems to their own. Not 
only could the SEA staff in these states receive 
research tailored to particular school improvement 
policy needs they faced, but they could then see 
how it might be put into action from other SEAs 
who were early implementers. 

Members of the core networks also worked together 
over time to develop, refine and use a set of ideas, 
protocol, tools, and frameworks. In these instances, 
the research-based blueprints or models were 
adapted through core groups for use in the context 
of a particular state. State C, for example, turned to 
the CII’s Handbook on Restructuring and Substantial 
School Improvement (Walberg, 2007) when 
redesigning its school improvement plans. The 
handbook contains a consolidated checklist of 
indicators for schools, districts and teachers to use to 


identify areas for improvement, a research-based 
guide to action that they found readily usable, and 
useful. The SEA, however, took a proactive stance 
with CII to help them transform these materials into 
what SEA staff considered a more focused and useful 
format. Working with their own school 
improvement coaches (who are retired educators), 
CII staff, and their regional comprehensive assistance 
center, State C created a more streamlined set of 
indicators, cutting the number from the Handbook 
by half. 

As this example suggests, the core networks included 
strong ties to networks of practitioners “on the 
ground” and in professional associations, as well as 
the research networks. Practitioner networks 
provided feedback on how improvement strategies 
were working in the field, what needed clarifying, or 
what could be changed. A core group member in 
State B described how they used meetings with a 
group of regional school district school 
improvement staff as a sounding board for their 
strategies: “because they’re in the schools doing 
school improvement with the local districts. And a 
lot of times we can say, ‘Okay, here’s what we’re 

thinking Is that too much? Is it not enough?”’ 

Staff in State B turned to professional membership 
organizations as well for practitioner input. 
Professional membership organizations were central 
to State C’s practitioner network, as were the 
comprehensive assistance centers. In contrast, the 
SEA in State A sought much of its practitioner 
advice from its districts and district networks. 

SEA staff also perceived information, decision- 
making, evolving improvement strategies and ideas 
to be more trustworthy and efficacious within the 
context of these core networks and collective work. 
For example, when asked if the people in his core 
work network have the expertise to find, and then 
use evidence to successfully improve low- 
performing schools, an office director reported: 


There is no one individual that holds all 
the information, which is why we have a 
group. . . all of those different people 
hold enough pieces that we can have 
conversations and share information 
across the table that can ... push us along 
... to that ideal goal at the end. 
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Resources such as “communal memory” created a 
collective sense of efficacy and supported core 
network members such that no one person had to 
know everything. When asked if and why she 
trusted research and other school improvement 
information exchanged within a core network, one 
influential office director in State B said: “Because 
we digest it together. And people challenge each 
other.” She continued with an example of a similar 
process: “We solve problems. What are we going to 
do about this? . . .And people bring in research and 
we’ll table things and [then] come back to them 
with the research, and then we’ll challenge the 
research.” An influential leader in State C noted: “I 
want to validate what [researchers are] saying. When 
you have those strong networks, you build upon that 
professional knowledge and practice.” 

Research Use 


The SEAs in our study valued and used research to 
inform the design of their school improvement 
frameworks, processes, tools, and other forms of 
school improvement assistance. As our earlier 
examples show, however, SEA staff gravitated to 
research that they perceived as relevant to their 
context, actionable, feasible, and helpful in addressing 
pressing problems of policy and practice, sometimes 
called “research designed for use.” In addition to 
searching for this kind of specific and context- 
relevant research, they also generated it by coupling 
research with practitioner knowledge. 

The states confronted similar problems and 
challenges in their work, even though many 
features of their systems of supports for low- 
performing schools varied. For instance, at 
different points in time all three searched for 
research to create school improvement frameworks 
and/ or planning processes that would be more 
effective in leveraging school or district change. 

All three sought research to help districts be more 
successful in managing the problems of their low- 
performing schools. And each used research to 
develop tools and processes that would aid the 
growing number of schools coming under the 
purview of accountability mandates. 

While foundational documents, such as the school 
improvement frameworks, could contain extensive 


references to the literature — State B’s framework 
contained 92 different citations, for example — SEA 
staff most often relied on research syntheses or 
research-based tools or strategies to provide very 
specific and concrete guides to action. For instance, 
State C drew on this type of research to overhaul 
their district and school improvement planning 
processes. The SEA viewed the plans submitted by 
low-performing sites as lacking a coherent theory of 
action, and premised on very localized notions of 
good practice rather than solid research. When they 
heard a presentation about the CII’s Handbook, they 
seized upon it to solve these problems. As described 
in the preceding section, the SEA created a more 
streamlined set of indicators that they believed 
would be more doable for sites. They also 
encouraged the CII to create a web-based platform 
using the indicators to help them more efficiently 
and cost-effectively monitor progress and interact 
with local educators. Building on this positive 
experience, the CII and the regional comprehensive 
assistance center continued to be a major source of 
research used by State C to develop new tools and 
processes, such as “change maps,” a process that the 
state could use to differentiate their technical 
assistance to sites. It was developed by the regional 
center and built on research from Banathy (1996), 
among others. 

State A brought together research with their own 
knowledge and local experience to develop a wide 
array of web-based school improvement tools and 
supports for its school and districts. The SEA 
collaborated with its urban school district network 
to identify common problems and develop guides 
for addressing these problems. They created a 
Professional Learning Community Guidance 
Document in conjunction with the National 
Institute for School Leadership (NISL) and a 
professional working group from districts and 
schools. The document, which provides guidelines 
for developing and strengthening instructional teams 
at the school level, includes references to research- 
based curriculum units from NISL and related 
research and SEA resources for each stage of the 
process. Similarly, the SEA’s school improvement, 
data, technology, and curriculum and instruction 
offices worked with a national consulting firm and 
five urban school districts to create a District Data 
Team Toolkit. Drawing on a research-based data- 
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driven inquiry and action cycle, the toolkit provides 
detailed modules and rubrics to help districts 
engage in inquiry and use data to inform district- 
level decisions. 

The states similarly pulled in research or evidence 
that had rich, descriptive details of practice that they 
could pass on to schools or delivery providers. One 
SEA staff member in State B noted that John 
Hattie’s book, Visible Learning: A Synthesis of Over 
800 Meta-Analyses Relating to Achievement (Hattie, 
2009), had “taken offlike wildfire” with their school 
coaches, in part because it presents a research-based 
dashboard for comparing innovations, a tool these 
providers could use to help their schools evaluate 
needs and select effective practices. SEA staff in 
both States A and B used the research-based 
Instructional Rounds in Education (City, Elmore, 
Fiarman, & Teitel, 2009) to focus their own and their 
districts’ school improvement strategies more 
specifically on classroom observations of instruction, 
student learning and academic content. State B was 
also trying to actively involve practitioners in 
recommending research-based resources. Educators 
could submit instructional resources aligned to the 
teaching and learning strand of the state’s framework 
on its website. But they had to also submit their 
assessment of the scholarship underlying the 
proposed resource, such as a judgment about the 
quality of evidence, how “seminal” the research was, 
and whether it was confirmed by other studies or 
experts. The state includes a link to this research 
evidence next to the recommended practice. Such 
attention underscores how deeply the reference to 
research has been embedded in the norms of SEAs. 

States also created their own original research by 
undertaking both formal and informal evaluations of 
their school improvement policies and programs. 

The extent to which the three SEAs conducted 
formal evaluations varied, however, depending on 
resources and internal capacity to select and guide 
external evaluations. SEA staff sometimes designed 
and conducted evaluations on their own, and other 
times they contracted with external partners. Under 
the direction of its Research Office, State A was the 
most involved in undertaking formal evaluations. 
State B contracted with national research 
organizations to evaluate their statewide system of 
support and their Title I School Improvement Grant 
program. In contrast, State C was more likely to use 


measures of academic improvement in identified 
schools and informal educator feedback to assess 
program success. But all of the SEAs drew on 
external evaluations of instructional programs or 
practices to create lists of acceptable or 
recommended programs or vendors to assist schools 
and districts. 

Summary and 
Implications 

In summary, SEA staff in our three study states 
actively sought and were receptive to research. 
Contrary to a uni-dimensional model of knowledge 
utilization, where research users are viewed as passive 
recipients of published research, research use was a 
multi-dimensional process in our sites. Multiple 
SEA staff reached out to multiple internal and 
external sources of research. Incorporating research 
into policy and practice was often a social process, 
where SEA staff worked with each other, 
practitioners, and external partners to make sense of 
research and adapt it to their local context. And key 
brokers of research inside and outside the SEAs 
facilitated the research search and incorporation 
process. Decision-makers were more likely to seek 
and use research designed for use than published 
academic studies to guide their actions. They also 
understood that research, particularly from 
recognizable and trusted sources, lent credibility to 
their efforts and motivated practitioners. 

Although the findings reported here come from an 
exploratory study of only three SEAs in one policy 
area, they shed light on ways that SEAs and 
policymakers can strengthen research-based 
knowledge use in their organizations. 

1 . SEAs should draw on the infrastructure outside 
their boundaries, such as technical assistance 
centers, state and national professional 
membership organizations, other professional 
networks, and universities, to access research and 
research designed for use. This action will, 
however, require SEAs to develop a culture of 
research use, and build the capacity to broker 
research search and incorporation, and to assess 
the underlying quality of the research and 
research designed for use produced by these 
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organizations. By cultivating multiple knowledge 
brokers within the SEA to access and circulate a 
diverse array of research, SEA knowledge 
networks will have access to a broader range of 
expertise and could also be less vulnerable to staff 
turnover. 

2. SEAs should also nurture, identify, and connect 
knowledge brokers in their agencies and in 
external organizations who work on common 
problems. SEA staff could develop and lead 
ongoing networks involving research 
organizations, practitioners, and their own staff to 
solve specific problems and advance state policy. 
The New York State Education Department’s 
Education Finance Research Consortium 
provides one example of this approach. Fostering 
working groups composed of influential SEA 
brokers, key research sources and practitioners to 
adapt generalized findings into more useable 
information in the context of particular state 
problems can facilitate the incorporation of 
research into policy and practice. 

3. Policymakers should encourage and support SEA 
evaluations of their own programs. These 
evaluations, particularly of the implementation of 
school improvement programs, provide critical 
feedback to agency staff. But SEAs often lack the 
human resources to design these studies and the 
fiscal resources to conduct them. 

Our study also raised more questions than we could 
answer, given the limited scope of our inquiry. First, 
researchers should study the use of research and 
other types of evidence in additional SEAs and other 
education policy areas to see whether the findings 
reported here generalize to other SEA contexts. 
Second, research should examine connections 
among patterns of internal or external information 
flow within an SEA, the number and type of 
information sources, the types of evidence people 
access and use in decision-making, and 
consequences for policy and practice. Third, there is 
a need to assess the quality of research acquired by 
SEA staff and underlying research designed for use 
products. While many of the products identified in 
our study were written by or cited national experts, 
sometimes research was added in a fairly superficial 
manner. Finally, there is a major need to strengthen 


the knowledge base, which includes supporting 
more varied types of research on policy 
implementation and effects in understudied areas of 
education policy. 
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